While workers can easily share gigabytes of project data on a local-area network (LAN) using standard file-server technology, such is not the case with workers in remote offices connected over wide-area networks (WANs). With respect to file sharing over WANs, standard file server protocols provide unacceptably slow response times when opening and writing files.
All major file-sharing protocols were designed for LAN environments where clients and servers are located in the same building or campus, including: NFS (Network File System, used for Unix/Linux environments), CIFS (Common Internet File System used for Windows environments), and IPX/SPX (Internetwork Packet Exchange/Sequenced Packet Exchange, used for Novell environments). The assumption that the client and the server would be in close proximity led to a number of design decisions that do not scale across WANs. For example, these file sharing protocols tend to be rather “chatty”, insofar as they send many remote procedure calls (RPCs) across the network to perform operations.
For certain operations on a file system using the NFS protocol (such as an rsync of a source code tree), almost 80% of the RPCs sent across the network can be access RPCs, while the actual read and write RPCs typically comprise only 8-10% of the RPCs. Thus 80% of the work done by the protocol is simply spent trying to determine if the NFS client has the proper permissions to access a particular file on the NFS server, rather than actually moving data. In a LAN environment, these RPCs do not degrade performance significantly given the usual abundance of bandwidth, but they do in WANs, because of their high latency. Furthermore, because data movement RPCs make up such a small percentage of the communications, increasing network bandwidth will not help to alleviate the performance problem in WANs.
Therefore, systems have been developed (called wide area file services (WAFS)) which combine distributed file systems with caching technology to allow real-time, read-write access to shared file storage from any location, including locations connected across WANs, while also providing interoperability with standard file sharing protocols such as NFS and CIFS.
WAFS systems typically include edge file gateway (EFG) appliances (or servers), which are placed at multiple remote offices, and one or more file server appliances, at a central office or remote data center relative to the EFG appliance, that allow storage resources to be accessed by the EFG appliances. Each EFG appliance appears as a local fileserver to office users at the respective remote offices. Together, the EFG appliances and file server appliance implement a distributed file system and communicate using a WAN-optimized protocol. This protocol is translated back and forth to NFS and CIFS at either end, to communicate with the user applications and the remote storage.
The WAN-optimized protocol typically may include file-aware differencing technology, data compression, streaming, and other technologies designed to enhance performance and efficiency in moving data across the WAN. File-aware differencing technology detects which parts of a file have changed and only moves those parts across the WAN. Furthermore, if pieces of a file have been rearranged, only offset information will be sent, rather than the data itself.
In WAFS systems, performance during “read” operations is usually governed by the ability of the EFG appliance to cache files and the ability to serve cached data to users while minimizing the overhead of expensive kernel-user communication and context switches, in effect enabling the cache to act just like a high-performance file server. Typically, the cache attempts to mirror the remote data center, so that “read” requests will be satisfied from the local cache with only a few WAN round trips required to check credentials and availability of file updates.
File management or explorer applications typically feature a graphical user interface that allows users to view the files and folders available either on a local hard drive or on a network share hosted on a remote file server. File systems are typically arranged in a hierarchy of folders with one or more files stored in the folders. Users can navigate within the folder hierarchy by selecting parent of child folders of the current folder. In WAFS systems, data regarding the folders and files stored within them are synchronized or refreshed between a local cache and a remote file server in order to provide users with a current view. In many WAFS systems, a local cache may request directory updates from remote file server or a cache server operating in connection with the remote file server as a user navigates a file system hierarchy. The directory updates typically entail the objects (files and folders) within a current directory, meta data relating to these objects, and security information, such as access control lists defining which users are permitted to access the files or folders. These update requests may spawn an appreciable amount of network traffic overhead when one considers that WAFS systems typically support a large number of remote users.